TeV gamma rays have been observed from blazars as well as from radio galaxies like M 87 and Centaurus A. In leptonic models, gamma rays above the pair production threshold can escape from the ultrarelativistic jet, because large Lorentz factors reduce the background photon densities compared to those required for isotropic emission. Here we discuss an alternative scenario, where very high energy photons are generated as secondaries from ultrahigh energy cosmic rays interactions in the cores of active galactic nuclei. We show that TeV gamma-rays can escape from the core despite large infrared and ultraviolet backgrounds. For the special case of Centaurus A, we study whether the various existing observations from the far infrared to the ultrahigh energy range can be reconciled within this picture.